Splitting of overlapping nuclei guided by robust combinations of concavity points

In this work, we propose a novel and robust method for the accurate separation of elliptical overlapped nuclei in microscopic images. The method is based on both the information provided by the global boundary of the nuclei cluster and the detection of concavity points along this boundary. The number of the nuclei and the area of each nucleus included in the cluster are estimated automatically by exploiting the different parts of the cluster boundary demarcated by the concavity points. More specifically, based on the set of concavity points detected in the image of the clustered nuclei, all the possible configurations of candidate ellipses that fit to them are estimated by least squares fitting. For each configuration, an index measuring the fitting residual is computed and the configuration providing the minimum error is selected. The method may successfully separate multiple (more than two) clustered nuclei as the fitting residual is a robust indicator of the number of overlapping elliptical structures even if many erroneous concavity points are present due to noise. Moreover, the algorithm has been evaluated on cytological images of conventional Pap smears and compares favorably with state of the art methods both in terms of accuracy and execution time.